-
Notifications
You must be signed in to change notification settings - Fork 763
Prompt optimizations #6664
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Prompt optimizations #6664
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull Request Overview
This PR optimizes the prompt instructions used for content moderation and topic classification in Mozilla's "{product}" support forums. Key changes include:
- Updating setup.cfg to ignore E501 in the prompt file.
- Overhauling the prompt in kitsune/llm/questions/prompt.py to clarify and improve the instructions for spam detection and classification.
Reviewed Changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated no comments.
File | Description |
---|---|
setup.cfg | Adds per-file ignore for E501 specifically for the prompt file |
kitsune/llm/questions/prompt.py | Refines the prompt for both spam and topic classification with updated instructions |
Comments suppressed due to low confidence (1)
kitsune/llm/questions/prompt.py:36
- [nitpick] Consider defining a common constant for JSON formatting instructions instead of using the same placeholder in multiple locations to improve maintainability and reduce duplication.
{format_instructions}
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
r+wc
I really like your changes, especially the adjustments to the topic prompt. My only recommendation is to remove the formatting instruction preambles that you added, since they're not needed.
kitsune/llm/questions/prompt.py
Outdated
|
||
|
||
# Response format | ||
Respond strictly with the following JSON structure: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The StructuredOutputParser.get_format_instructions()
takes care of adding the following preamble (prior to the JSON schema):
The output should be a markdown code snippet formatted in the following schema, including the leading and trailing "```json" and "```":
so this line can be removed.
kitsune/llm/questions/prompt.py
Outdated
- Look for keywords that align with topic descriptions and examples | ||
|
||
# Response Format | ||
Provide your classification using this exact JSON structure: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Same comment as above.
kitsune/llm/questions/prompt.py
Outdated
- `100` = Completely certain. | ||
4. Provide a concise explanation supporting your decision. | ||
|
||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Perhaps also remove this extra empty line?
|
||
```json | ||
{topics} | ||
``` | ||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Perhaps keep this blank line to add some separation from the next section?
d3c72df
to
8a3a4f4
Compare
No description provided.